(1) Field of the Invention
The present invention relates to the field of computer systems. More specifically, the present invention relates to the field of memory addressing method and systems within computer systems.
(2) Prior Art
Large computer systems today contain memory management hardware and address systems to provide software processes ("processes") with access to a defined range of physical memory (e.g., hardware volatile memory). Operating systems within the computer system provide data structures which define and maintain virtual memory spaces which can be made private to each process. The virtual memory space is then used transparently by the process to access data that is actually maintained within the physical memory space. The above data structures provide a mapping between the addresses of the virtual memory space used by the process and the addresses of the physical memory space used by the computer system's hardware memory management systems which actually maintain the data in the physical memory. A typical such computer system can provide memory mappings for as many as 500 or more processes simultaneously.
One such prior art memory management system (e.g., running within the VMS operating system) is shown in FIG. 1A. Two virtual address memory spaces are shown as 10a and 10b for process 1 and process 2, respectively. Also shown is a large physical memory address size 20. The physical memory address space 20 is typically realized in volatile memory (e.g., random access memory). According to the memory management system of FIG. 1A, within a 32-bit operating system, the allowed size of each virtual memory space 10a and 10b is approximately 2 gigabytes (Gb), with another 2 GB of memory space being reserved for the operating system. The operating system allocates a working set of physical memory to each virtual address space 10a, 10b. Working set 22a is allocated for virtual memory space 10a and working set 22b is allocated for virtual memory space 10b. In a 32-bit operating system, the typical size allowed for a given working set is about 10-20 megabytes (Mb), however, in some database applications a maximum of 0.25 Gb can be reserved. In a prior art 32-bit operating system, a maximum of 4 Gb of physical memory can be addressed. Therefore, the size of the working set is restricted by the operating system due to fairness concerns so that each process has sufficient access to the 4 Gb of physical memory.
A working set is divided into pages of physical memory, e.g., one page 24a is shown for working set 22a and one page 24b is shown for working set 22b. The size of a page is programmable, but one typical size is 8 kilobytes (Kb). When a process accesses data, it utilizes addresses within its virtual address space which are converted by a page table data structure into addresses within pages of the process's working set within the physical memory space 20. The operating system uses the page tables of each process to map pages of physical memory space 20 into the virtual address space (10a, 10b) that the process can access.
FIG. 1B illustrates three exemplary page table data structures 30a, 30b and 30n for process 1, process 2, and process n, respectively. The page table data structures 30a, 30b, and 30n are stored in the operating system's address space and maintain the mapping between a process's virtual addresses and the addresses of pages within the physical memory space 20. For each process, virtual addresses are received (e.g., over lines 32a, 32b, and 32n) and the appropriate page address of the physical memory space are output by the tables (e.g., over buses 34a, 34b, and 34n). Since many processes can operate simultaneously (e.g., 500 or more), many page tables can be simultaneously accessed and need to be simultaneously maintained.
In the addressing mechanism of FIG. 1A and FIG. 1B, many more physical memory pages can be assigned to a process than fit within the memory size of the working set for that process. In other words, the virtual memory size in the prior art memory addressing system, for a single process, is very much larger than the allocated working set size. In this case, as additional physical memory pages are needed by the process, they are retrieved from a non-volatile memory storage (e.g., disk 104 of FIG. 1A) and copied into the working set 22a, 22b for that process, overwriting other pages that might currently exist within the working set 22a, 22b. Conversely, if a page of a working set 22a, 22b is not currently required, it is stored in the non-volatile storage 104 to free up space within the working set 22a, 22b for other information. The above "swapping" of pages to and from the non-volatile storage 104 and the volatile working sets 22a, 22b is called "paging." Therefore, within the prior art memory addressing mechanism of FIG. 1A and FIG. 1B, disk input/output accesses are constantly being performed to flow data into and out of the working sets 22a, 22b because the virtual addressing spaces 10a, 10b of the processes are much larger than the working sets 22a, 22b. This constant I/O accessing (at approximately 30 accesses per second) reduces the overall performance of this memory addressing mechanism. Because this prior art addressing mechanism provides each process with a large amount of virtual memory space (e.g., 2 Gbin a 32-bit operation system) and a relatively small amount of physical memory space (e.g., 0.25 Gb), it is referred to herein as a "virtual memory centric" system.
Aside from the above performance disadvantage, the page table data structures 30a-30n of the prior art addressing mechanism of FIG. 1A and FIG. 1B consume a relatively large amount of operating system memory to maintain. For instance, the memory space required to maintain page table data structures 30a-30n for 500 processes, each process having 1 Gb of private virtual memory, requires at least 2 Gbof operating system memory. In a 32-bit operating system, the 2 Gb memory used by the data structures 30a-30n consumes half of the memory space available (4 Gb) to the operating system of the computer system. Such memory usage for mapping overhead is not an effective use of memory resources. The above disadvantageous operating system memory consumption results because the prior art addressing mechanism allocates a relatively large amount of virtual memory space for each process which can be as large as the physical memory space available, thus increasing the size of each page table data structure (e.g., of 30a-30n). Further, because the prior art statically allocates, on a per process basis, enough address space to map the entire physical buffer, depending on the operating system implementation, this can result in page table entry duplication on a per process basis, resulting in high system memory consumption.
The above performance and system memory consumption disadvantages are not alleviated or reduced within computer systems that contain 64-bit operating systems and/or 64-bit directly addressable physical memory spaces under the prior art addressing mechanism. Indeed, within these systems, the above performance and system memory consumption disadvantageous are exacerbated because the required sizes of the page table data structures per process are increased. Moreover, although 64-bit addressable physical memory provides a substantial amount of volatile memory storage, the prior art addressing mechanism still limits the size of the working set to 0.25 Gb per process. This is especially true for 32-bit operating systems having 64-bit directly addressable physical memory. Therefore performance disadvantages result due to heavy disk I/O operations. For 64-bit operating systems having 64-bit directly addressable physical space, the larger limiting factor concerns the immense size of the page table data structures within the operating systems address space.
To maximize system performance, database software and systems executing database software require access to very large memory (VLM) caches. On existing 16-bit and 32-bit operating systems, these VLM caches are limited by the maximum virtual address space recognized by the operating system, which can be at most 4 Gb, and is usually much less (e.g., 2 Gb). Therefore, in addition to the disadvantages noted above, the prior art addressing technique is also disadvantageous because it limits the amount of physical memory addressable to a process to 4 Gb in a computer system having a 32-bit operating system but 64-bit physical addressability. The prior art addressing mechanism is also disadvantageous because it requires a substantial amount of redesigning to operate with a computer system offering 64-bit physical addressability. Without the required redesign, a 32-bit operating system of the prior art addressing mechanism would inherently limit the amount of physical memory that could be efficiently addressed.
Accordingly, what is needed is an addressing system that eliminates the above performance and system memory consumption disadvantages of the prior art system. What is needed is an addressing system that can provide a particular process with access to more physical memory space than 0.25 Gb without requiring disk I/O operations during paging. Further, what is needed is an addressing system that can provide the above advantages within both 32 and 64-bit operating systems that have either 32 or 64-bit physical addressability. More particularly, what is needed is an addressing system that can provide processes within a 32-bit operating system with access to more physical memory space than 4 Gb within computer systems having a 64-bit physical addressability. The above is needed without requiring a substantial redesigning of the memory addressing system. The present invention memory addressing system provides the above features and others not necessarily recited above but clear within further discussions of the present invention herein.